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Abstract 

Assigning open textbooks in college and university courses can help students save money on increasingly 
expensive commercial textbooks, and recent research shows that this savings can often be achieved with 
little to no sacrifice in textbook quality or student learning outcomes. We add to this body of research by 
examining the use of an open textbook in an introductory physics course at a large research university in 
Canada that enrols approximately 800-900 students per year. In this course, the instructors revised an 
open textbook and combined it with other learning resources onto a single website, whereas more than 
one source of learning materials was used previously. We used the COUP framework to structure our 
analysis, focusing on cost, outcomes, use, and perceptions in relation to the open textbook assigned in the 
course. Through the use of a survey of students and data about student learning outcomes in the form of 
final exam and course grades, and shifts on the pre-/post- Colorado Learning Attitudes about Science 
Survey, we show that student savings by moving to an open textbook were accompanied by little change in 
learning outcomes. We also show that the vast majority of survey respondents perceived the open 
textbook to be of the same or better quality than commercial textbooks used in their other courses. 
Further, many of them appreciated the fact that the textbook was customized to this particular course— 
which is made possible by the use of a textbook with an open license. 
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Introduction 

The cost of textbooks at North American colleges and universities is a source of concern for many 
students, faculty, and staff. In August 2016 the U.S. Bureau of Labor Statistics reported that the price of 
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college textbooks has increased 88% since 2006, and is increasing at a faster rate than college and 
university tuition and fees (U.S. Bureau of Labor Statistics, 2016). Still, students are spending less on 
textbooks over time, not more: according to Kestenbaum (2014), student spending on textbooks went 
down from 2002-2013, and the average number of textbooks purchased per year by post-secondary 
students went down from about 13 to just over 8. Other research shows a significant percentage of 
students simply going without textbooks for some of their courses due to rising costs, as discussed below. 
Thus, as textbook prices go up, students seem to be responding by tiying to get by in at least some of their 
courses without the required textbooks, which could negatively impact their learning. In a survey of over 
22,000 post-secondary students in Florida, 38% said they believed they did worse in a course because 
they could not afford the textbook, and 20% said they had failed a course for that reason (Florida Virtual 
Campus, 2016). The high cost of textbooks is not just a financial concern, but a pedagogical one as well. 

One way to address the problem of students’ failure to access the required materials for their courses due 
to high costs is through the use of open textbooks and other open educational resources (OER). According 
to the William and Flora Hewlett Foundation (n.d.), OER are “teaching, learning, and research resources 
that reside in the public domain or have been released under an intellectual property license that permits 
their free use and re-purposing by others” (para. 2). OER can be texts (including open textbooks), videos, 
assessments, lecture notes, slides, or other learning materials. There is a growing body of research 
literature on open textbooks that strongly suggests students can save a significant amount of money if 
instructors assign open textbooks for their courses, while getting the same or better quality learning 
materials. Much of this research can be analyzed under what Bliss, Robinson, Hilton, & Wiley (2013b) call 
the COUP framework: measuring factors related to cost, outcomes, use, and perceptions of open 
textbooks. 

Research focusing on “cost” includes studies of how much students typically spend on textbooks, how 
much they save in particular courses by using open textbooks, and how the cost of textbooks affects 
students’ academic choices. The literature so far suggests that many students are paying at least $600 (in 
U.S. or Canadian currency) per year on textbooks. In a survey of over 22,000 students from 40 
universities and colleges in Florida, 53% of students reported that they spent over $300 on books in the 
Spring 2016 semester, and 33% spent over $400 (Florida Virtual Campus, 2016). In the Canadian context, 
Jhangiani & Jhangiani (2017) surveyed 320 undergraduates in various postsecondaiy institutions in 
British Columbia in 2015 and asked how much they had spent on textbooks in the last 12 months; the 
mean was about C$700 (median was $500). In a 2016 survey of about 1500 students from the University 
of British Columbia, nearly half of respondents (49%) said they spend at least C$500 on textbooks each 
year, and the mean reported cost per year for textbooks was $620 (So & Doering, 2016). 

The high cost of textbooks is, not surprisingly, affecting students’ academic behaviours, such as whether 
or not they purchase the required books for their courses and which courses they choose to take. Florida 
Virtual Campus (2016) asked over 22,000 post-secondaiy students whether the cost of textbooks had led 
them to do the following actions (among others) during their post-secondaiy studies (numbers below 
refer to the percentage of students who said they had done each action seldom, frequently, or often): 

• Didn’t buy the required book(s) for a course because of cost: 67%. 
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• Took fewer courses because of the cost of the book(s): 48%. 

• Didn’t register for a course because of the cost of the book(s): 46%. 

• Dropped a course because of the cost of the book(s): 26%. 

A report by Senack and the Student PIRGs (2014), using data from a survey of over 2000 students in 150 
colleges and universities in the US, provided similar numbers: 65% of student respondents reported that 
they had decided not to buy a textbook for one or more of their courses because of cost, and 48% said that 
the cost of textbooks had somewhat or significantly affected which or how many courses they were able to 
take each semester. Jhangiani & Jhangiani (2017) report that of over 300 students in British Columbia, 
54% said they had not bought a textbook for a course at least once due to cost, 27% said they had taken 
fewer courses, 26% said they did not register for a particular course because of the cost of the book(s), and 
17% said they had dropped or withdrawn from a course. So & Doering (2016) reports that 75% of 1500 
students at the University of British Columbia said they had gone without purchasing a textbook for at 
least one course. The research on cost reveals that the price of commercial textbooks is affecting students 
not just financially, but also in terms of their learning. 

Research on “outcomes” considers how the use of open textbooks and other OER affects students’ 
learning outcomes, including grades on exams or in courses, and retention rates. The literature suggests 
that in the majority of cases, students using open textbooks or other OER do just as well as or better than 
students who are required to purchase commercial textbooks or other course materials. Fischer, Hilton, 
Robinson, & Wiley (2015) studied a large sample of students from six colleges and four community 
colleges in the US. Students in the treatment group (N=io 87) used an open textbook or other OER in their 
courses, and students in the control group (N=9264) were asked to purchase traditional textbooks for 
different sections of the same courses. The authors found that the students using OER did just as well or 
better in most of the courses studied in terms of course grades and completion rates. Allen et al. (2015) 
studied students in a chemistry course at the University of California, Davis who were assigned a 
traditional textbook (448 students) and students in another section of the same course taught by the same 
instructor (478 students) that used Chemwiki fhttps: //chem.libretexts.org/l as a primary course resource 
(an openly licensed wiki site that can be used as a textbook). They found no significant differences in 
outcomes between these students, as measured through pre-and post-tests on content covered in the 
course, the same midterm and final exams, and the administration of the Colorado Learning Attitudes 
about Science chemistry survey (Barbera, Perkins, Adams, & Wieman, 2008), which asks about students’ 
“beliefs on learning chemistry, content of chemistry, structure of chemistry, and connection of chemistry 
to the outside world” (p. 942). 

Hilton (2016) provides a thorough analysis of these and other studies focusing on outcomes in the OER 
literature. In his synthesis of the data he concludes that “an emerging finding is that utilizing OER does 
not appear to decrease student learning” (p. 586). However, he emphasizes that there are significant 
limitations to such a synthesis of the research to date, because the OER may have been used differently in 
different courses and the design of some studies was too weak to claim any causal connection between 
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open textbooks and learning outcomes. Still, we can at least say that in the majority of courses studied, 
using an open textbook is not correlated with a reduction in grade averages or retention rates. 

Research on “use” includes data about how students use OER and how often, as well as in what format 
students access the materials. In addition, it can include empirical study of how the open licenses on 
learning materials allow for them to be used in new ways, such as instructors or students deleting material 
from, adding material to, or otherwise revising OER. Much of the research on use of open textbooks to 
date has looked at how often students use the books. Bliss, Robinson, Hilton, & Wiley (2013a) asked 490 
students from eight U.S. community colleges how often they tended to use textbooks in “a typical course,” 
and how often they used the open textbooks in the courses that assigned open textbooks (p. 23). Nearly 
75% of the students reported using textbooks 2-3 times per week or daily in “a typical course;” for the 
courses that used an open textbook, the percentage saying they used the book 2-3 times per week or daily 
was about the same. The California Open Educational Resources Council (2016) reports similar results 
from a survey of 351 students: most students said they used the open textbook they were assigned at about 
the same rate as compared to traditional textbooks in other courses. 

There are not many studies yet that discuss how students and faculty are taking advantage of the open 
licenses on open textbooks to alter them. In a survey of 127 post-secondary educators, Pitt (2015) reports 
that 25% said using an open textbook had “enabled innovation or changed their pedagogical approach” (p. 
146), but only a few explained in open-ended questions that they had revised an open textbook. From 
interviews with faculty who have adopted, adapted, or created an open textbook or other OER, Delimont, 
Turtle, Bennett, Adhikari, & Lindshield (2016) report that 12 out of 13 faculty said they preferred to teach 
with OER rather than commercial textbooks, and out of those, 11 gave customizability of OER as a reason 
for their preference. Neither study, however, focuses on how faculty are revising OER, and overall there is 
little research in the extant literature on the degree to which faculty or students are revising OER, how 
they are doing so, and what impact this may have on student learning outcomes. 

Research on “perceptions” considers how students, faculty or teachers, or others perceive open textbooks 
or other OER, such as their views about the quality of such resources. The literature shows that the vast 
majority of students report open textbooks to be of the same or better quality than the commercial 
textbooks they have used. Bliss et al. (2013a) reported that 94% of 490 students said they found open 
textbooks to be of equal or higher quality than traditional textbooks. Illowsky, Hilton, Whiting, & 
Ackerman (2016) reported on two surveys of students using an open textbook at a U.S. community 
college, one in 2013 of 231 students and one in 2015 of 94 students (the students surveyed in 2015 used a 
significantly-revised version of the textbook the students in 2013 had used). In the 2013 survey, 87% of 
students rated the quality of the open textbook to be the same or better when compared to traditional 
textbooks, and in the 2015 survey, 93% of student respondents did so. In a survey of over 300 students in 
British Columbia, 63% of respondents said the quality of the open textbook used in their course was 
“above average” or “excellent,” with another 33% rating the quality as “average” (Jhangiani & Jhangiani, 
2017). 

The literature on open textbooks so far suggests that: (1) students are spending a significant amount of 
money on commercial textbooks; (2) it is likely they could achieve the same or better learning outcomes if 
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their courses assigned open textbooks (keeping in mind the caveats above noted by Hilton, 2016); (3) 
students appear to be using open textbooks at the same rate as they use traditional textbooks; and (4) 
students overwhelmingly rate the quality of open textbooks to be just as high or higher than that of 
traditional textbooks. Overall, then, the picture emerging from the research suggests that assigning open 
textbooks in post-secondary courses is likely to provide the same benefits as commercial textbooks at no 
cost (or a fraction of the cost, for print versions). 

Our study adds an interesting and unique case to this literature for several reasons. First, the research 
literature on open textbooks to date has focused largely on U.S. post-secondary institutions, and our study 
provides survey data from students at a large, research-intensive university in Canada. McKerlich, Ives, & 
McGreal (2013) and Jhangiani, Pitt, Hendricks, Key, & Lalonde (2016) report on survey data from 
educators, staff, or administrators about use and creation of OER, but do not focus specifically on open 
textbooks and do not provide data from students. Our survey shares some similar questions to that used 
by Jhangiani & Jhangiani (2017) to gather data from students in British Columbia who have used open 
textbooks in their courses. In terms of providing empirical data from students in Canada on open 
textbooks, to our knowledge their article and ours are the first ones in the published literature so far. 

In addition, very little research on open textbooks has considered the specific ways in which faculty are 
revising open textbooks and whether this has any impact on learning outcomes. In the course we report 
on here, an open textbook was adopted primarily because the open license allows for revision. Two of the 
authors of this article participated in a significant revision of a first-year physics course (Physics too), 
involving moving all of the learning resources to a single website, and changing from a traditional 
publisher’s textbook accompanied by a paid online homework system to an open textbook and homework 
questions embedded in the course website. Unlike some studies that focus on OER or open textbooks 
generally, such as McKerlich et al. (2013), Jhangiani et al. (2016), Bliss et al. (2013a), Fisher et al. (2015), 
and Delimont et al. (2016), we provide a very specific description of how the open textbook used in the 
course we are studying has been adapted to fit into that course. 

Using data from the 2015-2016 academic year, we studied students’ experiences with and perceptions of 
the new website and textbook for Physics 100. We also examined student learning outcomes in the form of 
final exam grades, course grades, and the Colorado Learning Attitudes about Science Survey (CLASS; 
Adams, Perkins, Podolefsky, Dubson, Finkelstein, & Wieman, 2006), in the years before and after the 
revision to the course. 


Context 

This study reports on the use of an open textbook in Physics 100: an introductory, algebra-based physics 
course at a large, Canadian research university. As of November 1, 2016, over 43,000 undergraduate 
students and 9600 graduate students (full time and part time) were registered at the University of British 
Columbia in Vancouver (UBC Planning and Institutional Research, 2017). Physics 100 is taken by 800- 
900 non-physics majors annually that did not take grade 12 physics in high school, and it is a required 
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course for 99% of the students taking it. The changes to the course described above, including the 
adoption of an open textbook, were first implemented during the Fall 2015 and Spring 2016 terms. 
Physics 100 in the Fall term is a face-to-face course; in the Spring term it is a fully online, distance course. 

Data from 2013-2016 show that the vast majority of students in Physics 100 are in their first year (average 
80%), and an average of 65% are in the Faculty of Science, 10% in the Faculty of Arts, and 25% in other 
faculties. In addition, on average 66% of students in the course identify as female and 36% as male. In 
comparison, at the University of British Columbia-Vancouver, 55% of undergraduates identify as female, 
31% are in the Faculty of Arts and 19% are in the Faculty of Science (UBC Planning and Institutional 
Research, 2017). 

The course designers chose to use the open textbook College Physics ( https://openstax.org/details/college- 
physicsl , published by OpenStax. An electronic version of the textbook was customized, re-formatted into 
weekly reading assignments, and integrated into the course website on edX edge (edge.edx.org). 1 This 
website also contains all other course resources and replaces the previous commercial online homework 
system and the Learning Management System. The end-of-chapter textbook questions as well as previous 
exam questions were imported into edX and coded into a question database to be used for online 
homework and as a test bank for frequent testing. There is also a space for in-class materials, such as the 
worksheets used in lectures and tutorials. These materials were previously created by instructors and did 
not change after the revision. Furthermore, the edX web site allows the seamless integration of a 
discussion forum (piazza.com) that was already used before the revision. Table 1 provides a list of all 
course elements and materials. 

Table 1 


Overview of Physics 100 Course Elements 


Course activity 

Required materials 

(Pre-) Reading 

Textbook, instructions, quiz 

Lecture 

Worksheets, concept questions (“clickers”) 

Tutorial 

Worksheets 

Discussions 

Online Forum 

Lab 2 

Lab worksheets, common materials (rulers, stopwatches, ball, etc.) 

Homework 

Online homework (problem questions) 


From a pedagogical perspective, the instructors were aiming at creating a more coherent appearance of 
the course resources that helps students make connections between the in-class activities and the 
homework. The instructors found that in previous iterations of the course, the pre-class reading 
assignments did not prepare the students well enough for class. This is of particular importance because 
the course is delivered in a “flipped” format in which students read the textbook (or watch a video) to 
learn basic terminology, definitions, and simple facts before coming to class. Class time is spent on sense- 
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making and applying this knowledge, and it is therefore very important that the students come prepared 
to class by having done the reading. Before the revision to the course, the reading assignments consisted 
of (a) reading instructions (what sections to read, which examples and concepts to focus on, etc.) that 
were posted on the online homework system, (b) reading the textbook, and (c) answering quiz questions 
that were also posted on the online homework system. A study by Heiner, Banet, and Wieman (2014) has 
shown that such a supported format is successful in getting most students to read the textbook regularly 
(typically 80%). However the study does not address the quality of the reading: for example, how much 
students understand. Anecdotally, we find that reading levels may have declined in recent years as more 
courses adopt a “flipped” format. This is supported by the rates in which simple reading questions are 
correctly answered in class (using clickers). The new reading assignments in Physics too have the quiz 
questions embedded into a customized version of the text that is now broken up and organized by weeks 
and only has the textbook sections that are relevant to the classroom activities for each week. Reading 
instructions are therefore no longer needed and students are more likely to read the text due to the 
proximity of quiz questions and relevant text. 

As mentioned above, the end-of-chapter questions of the open textbook now replace the commercial 
online homework system that was previously used. There are two main reasons for this switch: cost and 
administration. Online homework systems are widely used in first-year science courses and major 
publishers offer packages that include the textbook and an online homework system customized for the 
textbook. The cost difference between such a package and the textbook alone or the online homework 
system alone is so small that adopting just the open textbook and keeping the commercial online 
homework system would not have saved the students much money. The instructors therefore decided to 
create an online homework system directly in the course web page by importing the end-of-chapter 
questions of the open text. The other reason for this decision is in avoiding administrative overhead for 
students and instructors. Using a commercial online homework system means adding another online tool 
that students have to learn how to use, register, and pay for. At our institution, first-year courses in 
chemistry, physics, and math all use different online homework systems in addition to the learning 
management system. This can lead to confusion, technical difficulties, and delays in getting the students 
up and running on these systems. By embedding the question database in the web page we avoid these 
issues. 


Research Questions and Method 

We used the COUP framework to design research questions for this study. 

Cost 

• How much do students in this course report usually spending on traditional textbooks, and how 
much did they save with the open textbook? 

• What is the impact of the cost of traditional textbooks on the academic behaviours of students 
who responded to our survey? 
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Outcomes 

• Are the changes implemented in this course, including the use of an open textbook, correlated 
with any changes in student grades on the final exam, grades in the course, or results on the 
CLASS survey? 

Use 

• Did the students in this course report using the open textbook more, less, or about the same as 
they report usually using textbooks for their courses? 

Perceptions 

• How did students perceive the quality of this open textbook, as compared to traditional textbooks 
in other courses? 

We addressed these questions using data from three sources: (1) a survey of students who had taken 
Physics too in Fall 2015 or Spring 2016, (2) final exams and course grades for students in both terms, and 
(3) results from the Colorado Learning Attitudes about Science Survey (CLASS) for physics. 

We administered the survey to students who had taken Physics too in Fall 2015 (face-to-face course) or 
Spring 2016 (online course); the questions and results of the survey is publicly available here: 
https://osf.io/928wb/ . It consisted of a mix of multiple-choice, many-choice, Likert-type, and open- 
ended questions. A link to the online survey was sent via email in April 2016 to students who had taken 
the course in the Fall of 2015, and in May 2016 to those who had taken it in the Spring of 2016 (after final 
course grades were submitted). Table 2 shows survey response rates. 3 

Table 2 


Response Rate for the Survey, out of the Number of Students who Completed Physics 100 


Term 

Completed course 

Completed at least half 
of survey 

Completed entire 
survey 

Fall 2015 

755 

140 (19%) 

134 (18%) 

Spring 2016 

56 

12 (21%) 

9 (16%) 

Total both terms 

811 

152 (19%) 

143 (18%) 


Of the respondents, 85% were first-year students (compare to an average of 80% in the course since 
2013), 71% identified as female, and 27% identified as male (compared to an average of 65% female and 
35% male from years 2013—2015). The majority (66%) said that their primaiy area of study was in the 
sciences (e.g., Chemistry, Biology, Physics), while 13% said it was in the health sciences, and the rest were 
spread between disciplines in the social and applied sciences (5% chose “I haven’t decided yet”). From 
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2013-2015, 65% of students in the course were in the Faculty of Science, and 10% in the Faculty of Arts. 
Despite the fairly low response rate we can say that the demographics of the survey respondents 
correspond roughly to those of the course itself except that few students on the survey were from the 
Faculty of Arts. 

We used descriptive statistics in our analyses of the multiple-choice and Likert-style questions on the 
survey. For analysis of the answers to open-ended questions, we used an open coding process: we 
generated codes from patterns found in the data itself. Two of the authors of this study separately coded 
the qualitative data using the same set of codes and either agreed with the other upon seeing both sets of 
codes or discussed disagreements to come to a final conclusion. We report below on statistical methods 
used to evaluate whether the changes to the course resulted in changes to student learning outcomes. 


Results 


Cost 

As some previous studies in the literature review above have done, we asked students how much they 
typically spend on textbooks for their courses (see Table 3). The last column on the right lists the average 
number of courses taken per year by students who reported each dollar range. 

Table 3 

Reported Amount Spent on Textbooks and Average Courses Taken per Academic Year (Fall, Spring, and 
Summer Terms Combined ) 


Amount spent on 
textbooks/year 

Number 

% of N 

Average number of 
courses taken/year 

Less than $200 

17 

11% 

8-3 

$200-$400 

65 

48% 

9.0 

$40i-$6oo 

42 

28% 

9-7 

$6oi-$8oo 

22 

18% 

10.6 

$8oi-$iooo 

1 

1% 

10.0 

$1001-$1200 

3 

2% 

9-3 

More than 1200 

1 

1% 

10 


Note: N=i5i; amounts are in Canadian dollars. 
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Most respondents reported typically spending between $200 and $400 per academic year on textbooks 
(48%), though if we combine those who said they spend between $400 and $800 per year that’s another 
46%. These numbers are somewhat surprisingly low, since, for example, if students who spend $200- 
$400 per year are taking an average of nine courses a year, that means they could be spending between 
$22 and $44 per course on textbooks. We consider possible reasons for our low numbers in the 
Discussion section of the article, below. 

For Physics too, the learning materials used in previous years cost students about C$100. The instructors 
were using a customized version of a textbook that included only the sections covered in the course, which 
already saved students money over the standard version. The $100 cost also included the price of 
students’ access to the online homework system. For the 8s6 4 students who were enrolled in Physics 100 
during the 2015-2016 academic year, that’s over $85,000 saved compared to what they would have spent 
if they had to purchase the $100 course materials used in previous years. 

We also asked about the impact of textbook costs on students’ academic choices (see Figure 1). 


Dropped or withdrew from a course because 
the textbook was too expensive 

Took fewer courses than I'd like 
because I can't afford the books 

Registered for a different section of 
a course because the textbook was cheaper 

Did not purchase a required textbook 
for a course because of cost 



6 25 50 75 100 

Percentage of Respondents 


Action 
I Never 

I Rarely or Sometimes 
Often or very often 


Figure 1. How often respondents have taken certain actions as a result of textbook costs. N=iso. See 
Appendix A for percentages for each category. 


Adding up the percentages for “rarely,” “sometimes,” “often,” or “very often,” 57% of students stated that 
they did not purchase a textbook for at least one course because of the cost, 25% said they had registered 
for a different section of a course, 19% said they had taken fewer courses than they would have liked, and 
16% said they had dropped or withdrawn from at least one course. 

Outcomes 

To address the question of whether the changes in the course and adoption of the open textbook affected 
students’ learning outcomes, we examined student final exam and course grades and student attitudes 
measured with the CLASS diagnostic survey that is well established in the physics education community. 
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Measuring student performance with final grades is usually not a reliable method for longitudinal 
comparisons due to changes in the weighting of different course components. Similarly, final exam results 
are influenced by variability in question difficulty and it is fairly difficult to establish an overall level of 
difficulty on an absolute scale. For instance, over the period of 2008-2015, we observed final exam 
averages between 54 to 64%, or when excluding two outliers, 60-63%. In this case however, the exams are 
very similar and are designed by the same instructors. The exam score average for Fall 2015 was 63.1% 
and was slightly higher than the one in Fall 2013 (61.4%, p=0.04) and the one from Fall 2014 (61.2%, 
p=0.02). The distributions were compared with a Wilcoxon rank sum test with continuity correction—p 
values quoted above have not been corrected for multiple comparisons. Given the marginal absolute 
change (2% grade points) in final exams scores, we can say with some confidence that student 
performance on the final exam is at least as good as in previous years. 

For student course grades, we considered both the distribution of grades into As, Bs, Cs, and Ds, and also 
the percentage of students who failed the course (“fail” is a grade of F). Figure 2 shows these numbers for 
Fall 2012 to Spring 2016. The changes to the course, including the adoption of the open textbook, were 
made in the Fall of 2015. 



2012 Fall 2013 Spring 2013 Fall 2014 Spring 2014 Fall 2015 Spring 2015 Fall 2016 Spring 


Figure 2. Percentages of students earning Grades A-F in Physics too, from Fall 2012 to Spring 2016. Fall 
terms are face-to-face versions of the course, and Spring terms are online, distance versions. See 
Appendix B for percentages in each category. 

Grade distributions in Fall 2015 were very similar to previous Fall semesters, before the changes to the 
course were made. In Spring 2016 grades of “A” (51%) went up quite a bit compared to Spring 2015 (36%). 
However, non-parametric testing of differences in the final grade distributions showed no significant 
changes between the terms in which students used an open textbook and the previous iterations of the 
course. As discussed above, final grades are influenced by many factors, and the fact that there are no 
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significant changes in grade distributions allows us only to conclude that the introduction of the open 
textbook caused no large changes in the final grade. 

Finally, we used the CLASS attitude survey to compare the student attitudes and beliefs about physics and 
about learning physics to previous years. Students choose how much they agree or disagree with 42 
statements on a five-point Likert scale. The results are then evaluated and compared to the answers of 
discipline experts. The CLASS is administered as a pre-/post- survey during the first and last week of the 
course and shifts in responses are linked to the course instruction. Average positive shifts in student 
responses are interpreted as students’ views becoming more expert-like. Negative shifts indicate a more 
novice-like attitude after taking the course. The survey questions fall into eight empirically determined 
categories (Adams et al 2006): real world connections, personal interest, sense making/effort, conceptual 
connections, applied conceptual connections, problem solving general, problem solving confidence, and 
problem solving sophistication. 

The results from 2015-2016 are overall similar to previous years, but slightly more negative (see Table 4). 
Performing statistical tests using one-way ANOVA shows that the changes are statistically significant in 
the problem-solving category (F = 4.16 > F cr it. = 2.61; p < 0.05) while there are no significant changes in 
the shifts for all questions combined (F = 1.36 < F cr it. = 2.61) and in the overall post course results (F = 
0.82 < Fcrit. = 2.61). 

Table 4 


Results of CLASS Survey 


Year 

CLASS -PS_Shift 

CLASS-All_Shift 

CLASS-ALL_Post 

2010 

4.1 ± 1.4 

-0.9 ± 0.9 

51.0 ± 1.1 

2011 

4.2 ± 1.1 

0.5 ± 0.7 

52.5 ±0.9 

2012 

3-7 ± 1-4 

-0.3 ± 0.8 

54-5 ± 1-0 

2015 

-1.9 ± 1.4 

-1.7 ± 1.1 

52.2 ± 1.1 


Note. Pre-post shifts for the general problem-solving category are listed in column 2 and shifts for all 
survey questions are presented in column 3. The fourth column shows the post course averages for all 
survey questions categories. 5 The “year” in column 1 includes Fall and Spring terms (e.g., 2015 includes 
Fall 2015 and Spring 2016 terms). 

Negative shifts are typically observed in first-year physics classes and very few courses have been reported 
with positive shifts (Redish & Hammer, 2009). Nevertheless, we have been able to obtain consistent 
positive shifts in the problem solving categories in previous years. That these shifts are now slightly 
negative is currently actively discussed by the instructors, and possible reasons for this are given in the 
Discussion section below. 
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Use 

In two separate questions, we asked students how often they tend to use textbooks in a typical course, as 
well as how often they used the open textbook in Physics too (see Table 5). 6 

Table 5 

Reported Usage Rates of Textbooks 


Usage rate 

Typical course 
(N=i 5 i) 

Open textbook for Physics 100 
(N=i 43 )) 

Never 

1% 

4% 

2-3 times per term 

7% 

6% 

2-3 times per month 

27% 

27% 

2-3 times per week 

58 % 

57 % 

Daily 

7% 

5 % 


The patterns of usage for both questions are very similar, with over 55% of students saying they typically 
use their textbooks, and used the Physics too textbook, 2-3 times per week. 

We also asked students to explain why they did or did not typically use textbooks very often for their 
courses, as well as for Physics too. The most commonly reported reasons in these open-ended questions 
were similar for both. Common reasons for why students do typically use textbooks often in their courses, 
and for Physics too, were: to do assigned pre-readings, to help understand the material better than they 
could just from going to class alone, and to prepare for quizzes and exams. The most common reasons for 
not using textbooks in a typical course were: one can get all the information one needs from class, and 
much of the material in textbooks is not relevant to exams. But a smaller percentage of students gave 
these reasons for not using the Physics too textbook: 23 respondents (18% of 130 who answered this 
question) said that in typical courses, they can get what they need better from class than from the book, 
whereas only five respondents (5% of 122) said this for the Physics too textbook. In addition, 10 students 
(8% of 130) said that in typical courses, much of the material in textbooks is not relevant to exams, but 
only four respondents (3% of 122) said that was the case for the Physics too textbook. These results could 
be due to the fact that because the Physics too textbook had an open license, the instructors were able to 
customize it by only including those parts of the text that were directly relevant to the course. 

Perceptions 

When asked how they would rate the quality of the textbook used in Physics too as compared to textbooks 
they have used in other courses, 93% of 143 respondents said it was about the same or better than 
textbooks in other courses (72% said it was about the same, and 21% said it was better). Students were 
invited to explain their answers in open-ended responses, though only 33 of them did. Of those, seven 
(21% of those who explained their answers) said that they appreciated the fact that the text was 
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customized to the course. Three respondents (9%) said they found the book easy to understand and that it 
explained topics well, but another three said that they thought the text did not explain the material well 
enough. Those three students were in the distance education course, and said that the text was not enough 
to explain what they needed to know without there being in-person class meetings. Finally, five students 
who said the quality was worse than that of their other textbooks said this was because of the format of 
the online book, such as that it was hard to navigate or that scrolling through PDFs was bothersome. 

We also asked students how important various aspects of the Physics too textbook were to them, and 
three answers stood out as most important: the book was customized to the course, it was free, and it 
could be accessed anywhere with an internet connection (see Figure 3). 

The readings were customized 
to this particular course 

The Physics 100 textbook did not 
cost any money 

I could access the textbook anywhere 
with an internet connection 

The textbook could be read on my phone 
or other mobile device 

I could download the readings and read them 
even without an internet connection 

I could download the readings 
and print them 

0 25 50 75 100 

Importance Rating / % 

Figure 3. How respondents rated the importance of various features of the Physics 100 open textbook. 7 
N=i43. See Appendix C for percentages for each category. 

The vast majority of the student respondents (94%) appreciated that the textbook was customized to their 
particular course. Unsurprisingly, then, when asked on a different survey question whether or not they 
agreed that the textbook was relevant to the course, many respondents said it was: 78% of 142 
respondents stated that they either somewhat or strongly agreed that the readings were linked directly to 
lectures, labs or other class activities, and 66% at least somewhat agreed that the readings were useful in 
preparing for exams. 

As another way to gauge their perceptions of the open textbook for Physics too, we asked students to rate 
their agreement with the following statement: “I would have preferred to purchase a traditional textbook 
for this course rather than using the free online textbook.” The majority of students disagreed, saying they 
would rather use the free, open textbook: 64% of 143 respondents disagreed somewhat or strongly with 
that statement, 18% said they neither agreed nor disagreed, and 18% said they somewhat or strongly 
agreed that they would rather have purchased a traditional textbook for the course. 

Among the 91 students who explained their answers, the most commonly-reported reasons given for not 
wanting to purchase a traditional textbook were that the online textbook was free (65%), that traditional 
textbooks are heavy to carry around (12%), and that the textbook was customized to fit this particular 
course (8%). Among the 25 out of 143 respondents who said they would have preferred to purchase a 



91 


The Adoption of an Open Textbook in a Large Physics Course: An Analysis of Cost, Outcomes, Use, and Perceptions 

Hendricks, Reinsberg, and Rieger 


traditional textbook instead of using the free, online one for Physics too, only 15 explained their answers. 
Among those, 14 gave reasons related to preferring reading print versus online books (e.g., because they 
do not like reading on a screen, or they like to highlight text in print books), not because of the quality of 
the textbook. Only one gave a negative comment about the quality of the textbook. 


Discussion and Limitations 

The results of this study strongly suggest that the students in Physics too were able to save money by 
using an open textbook that allowed for customization to the course and embedding in a consolidated 
website, while getting similar quality as they would from a commercial textbook. The vast majority of the 
survey respondents judged the open textbook to be the same or better quality than textbooks used in their 
other courses (93%). This result is similar to those in other published research on open textbooks. 
Compared to the one other survey so far of Canadian students that asked a similar question, or results on 
whether students would have preferred to purchase a traditional textbook rather than using the open 
textbook, are slightly more positive. 64% of respondents in our survey said they would not have preferred 
to purchase a traditional textbook instead of using the OpenStax book; 56% of respondents said this in a 
survey of 320 students in British Columbia (Jhangiani & Jhangiani, 2017). 

Students in our study reported spending less on textbooks per academic year than those in previously 
published studies, especially considering that the Canadian dollar has been worth less than the U.S. dollar 
for the past five years (XE Currency Charts (CAD/USD), 2016). Florida Virtual Campus (2016) reports 
that between 45-50% of students say they spend over US$300 per term ($600 per year), whereas 48% of 
our student respondents reported spending between C$200 and C$400 per year, with only 22% saying 
they typically spend over $600 a year on books. In a survey of post-secondary students in British 
Columbia, respondents reported spending a mean of about $700 and a median of $500 in the last 12 
months on textbooks (Jhangiani & Jhangiani, 2017). Our numbers are somewhat closer to a survey of 
1500 students at the University of British Columbia (So & Doering, 2016): 49% reported spending at least 
C$500 per year on textbooks whereas 50% of students on our survey reported spending at least $400 per 
year (our closest comparable number). One possible reason the reported numbers on our survey are lower 
than some other results is because 85% of our survey respondents were in their first year, so they did not 
have a great deal of experience to draw from in giving their answers (they did, however, take the survey at 
the end of their first year, after purchasing textbooks for two terms). The data from earlier literature does 
not split out reported spending by first-year students versus data from other students. 

Our data about how textbook costs affect students’ academic choices are lower than numbers from studies 
of U.S. students. In our survey, 57% of respondents reported not buying a required textbook for at least 
one course because of cost, 25% reported registering for a different section of at least one course, and 19% 
said they had taken fewer courses. In comparison, data from earlier studies (Florida Virtual Campus, 
2016; Senack & The Student PIRGs, 2014) shows over 60% reporting not purchasing a textbook for at 
least one course because of cost, and over 40% saying that the cost of textbooks affected their choice of 
courses as well as the number of courses they could take. Our data is closer to that from students in 
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British Columbia, however: Jhangiani & Jhangiani (2017) report that 54% of students said they had not 
purchased a textbook because of cost, 26% said they had chosen not to register for a specific course 
because of the textbook cost, and 27% had taken fewer courses. Still, our numbers are lower than those 
from a larger survey of students at our institution (So and Doering, 2016), which reported that 75% of all 
1500 respondents, and 70% of those in their first or second year, had gone without a textbook for at least 
one course because of cost. In comparison to these numbers at our institution, it seems the first-year 
students in our survey are skewed towards the lower end of being affected by textbook costs in terms of 
purchasing textbooks or choosing courses. 

Looking at student learning outcomes, we find overall that final exam marks and course marks after the 
changes to the course (including the adoption of the open textbook) are comparable to data from the 
previous three years in Physics too. We see no significant change in the overall post-scores and overall 
shift-scores of the CLASS survey, while a small negative shift in the “problem-solving” categoiy in 2015- 
2016 occurred. Possible reasons include the homework incentive and feedback as well as a change in the 
tutorial structure. Previously, students received marks for the homework that was worth 10% of the 
overall grade, while students received no marks for doing the homework in 2015-2016. In the future, we 
will again reward doing the homework with a few percent of the final grade. It is also possible that the 
current feedback on the homework is not sufficient. The previous commercial homework system had 
tutorial-type questions that provided hints and students could get feedback on important steps using 
these hints. Our homework system only provides correct/incorrect feedback, but we will enhance our 
homework system in the near future. For the Fall 2016 term, animated worked examples were inserted in 
the homework sets, which have been shown to enhance learning. In addition, the instructors have 
developed tutorial-type questions with step feedback, similar to commercial homework systems used 
previously. Finally, there may also be an effect due to the textbook. Although the open text features 
student-centered learning support such as problem-solving strategies and worked examples, it not as 
detailed as the previously used commercial text. However, we do not think that the textbook has a 
significant influence on the students’ problem solving ability and confidence as compared to the course 
components that specifically focus on problem solving, in particular the homework and the tutorial 
sessions. 

Regarding use of the open textbook, similar to results from other published studies we did not see a 
significant difference between how often students reported using textbooks in a “typical course” versus 
how often they used the textbook in Physics too. Combining this with the data about their perceptions of 
the quality of the textbook suggests that they found the textbook useful for their course work while also 
having relatively few complaints or concerns about it. 

This study is limited by its relatively small sample size compared to the number of students enrolled in 
Physics too, as well as by the fact that the data is from one course at one university only. However, our 
results are largely consistent with previous research on open textbooks, with the exception of how much 
our students reported typically spending on textbooks per academic year and how costs affect their 
academic behaviours such as choosing which section of a course to take based on textbook cost. Our study 
is also focused on a course in which several changes were made, in addition to adopting an open textbook, 
including that the book was customized to fit this course and all course materials were consolidated on a 
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single website. While this makes the results of our study less applicable to courses that adopt an open 
textbook only, it also provides evidence of the value of customizing an open textbook and embedding it 
into a website with homework and quiz questions. 


Conclusion 

The results of this study add to and mostly support the existing literature on open textbooks in terms of 
cost, learning outcomes, student use, and student perceptions. We have added to the so far limited data 
on students in the Canadian context, and our study focuses on a case of not just adopting an open 
textbook, but also significantly adapting it to fit a particular course. 

However, a project like ours is not without cost: a significant amount of effort went into the project, in 
particular into the creation of the question database and the test bank. Four graduate physics students 
and two instructors worked over the summer months to get the resources ready; approximately C$20,000 
from a teaching and learning grant from our university have been spent to date to pay the graduate 
students. The instructors contributed a significant amount of time and effort to secure funding, guide the 
project, select materials, get familiar with edX, and post materials, troubleshoot, etc. Additionally, one of 
the graduate students and the university’s information technology staff provided technical help. Although 
the open text is free, integrating it into a coherent course package comes with significant development 
costs. 

The literature on open textbooks related to cost focuses on cost savings to students, but it’s important to 
keep in mind the possible costs for faculty and institutions in terms of time and support when using open 
textbooks. Even adopting an open textbook “as is” comes with a cost insofar as any change in a textbook 
for a course means at least some extra time is needed to connect the new text to the course. Our study 
suggests that taking advantage of the open license of open textbooks to make significant revisions can be 
pedagogically useful and appreciated by students, but it may require a good deal of time and resources. 


1 EdX is a platform that has a number of Massive, Open, Online Courses (MOOCs), but there is also a platform for courses that are 
accessible only with a registration called edge.edX. The Physics 100 course, though the course materials are on edX, is not a MOOC. 
EdX is used as a Learning Management System for students and faculty in the course. 

2 Another change to the course is that the labs are now completed at home, rather than in person. However, the labs mostly build 
experimental data skills and we do not believe this change is relevant to the study of the open textbook presented in this article. 

3 This survey was incentivized: students in both the Fall 2015 and Spring 2016 courses who were invited to participate in the survey 
were eligible to enter a draw for one of four C$25 gift cards to either the bookstore or a coffee shop on campus (their choice). 

4 885 students registered for and completed at least some of the course; 74 students withdrew or otherwise did not complete the 
course during the normal term, leaving 811 students who completed the entire course (as reported in Table 1, above). Those who 
withdrew or otherwise did not complete the course would have left after the official add/drop period, and thus may not have been 
able to return their textbooks for a full refund. 

5 For experts: we just show here the shifts for the “favorable” category. The “unfavorable” shifts are highly correlated with these. We 
just show here the general problem solving category. The shifts for problem solving confidence and sophistication are similar. 

6 We did not define “typical course” on the survey; the question was: “For a typical course, how often do you use the required texts?” 
However, since most of the students who responded to the survey reported majoring in the sciences, then a “typical course” for them 
is probably a science course. 

i There could be a close connection between “somewhat important” and “of average importance.” The latter is meant to signify 
something like “neutral.” The answer range for this question was: “very unimportant,” “somewhat unimportant,” “of average 
importance,” “somewhat important,” “very important.” We took this wording from another survey of post-secondary students in 
Canada, and in future will change the middle term to distinguish it further from “somewhat important.” 
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Appendix A 

Data for Figure 1: How Often Respondents Have Taken Certain Actions as a Result of 
Textbook Costs 


Actions taken because of textbook costs 

Never 

Rarely or 

sometimes 

Often or 
very often 

Dropped or withdrew from a course because the 
textbook was too expensive 

84.0% 

14.7% 

1.3% 

Took fewer courses than I’d like because I can’t 

afford the books 

81.3% 

16.7% 

2.0% 

Registered for a different section of a course 
because the textbook was cheaper 

74-7% 

22.7% 

2.7% 

Did not purchase a required textbook for a course 

because of cost 

43-3% 

43-3% 

13-3% 


Note: N=i 50 
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Appendix B 

Data for Figure 2: Grades Distribution in Physics 100, Fall 2012-Spring 2016 


Year and term 

A 

B 

C 

D 

F 

Fall 2012 (N=728) 

44% 

41% 

6% 

1% 

2% 

Spring 2013 (N=34) 

41% 

21% 

12% 

15% 

3% 

Fall 2013 (N=742) 

35% 

45% 

11% 

2% 

7% 

Spring 2014 (N=33) 

36% 

30% 

30% 

0 

3% 

Fall 2014 (N=75 o) 

28% 

40% 

20% 

2% 

10% 

Spring 2015 (N=67) 

36% 

39% 

12% 

3% 

11% 

Fall 2015 (N=755) 

37% 

38% 

16% 

2% 

7% 

Spring 2016 (N=56) 

51% 

18% 

16% 

11% 

4% 


Note: The Fall version of Physics 100 is a face-to-face course, while the Spring version is a distance course. 
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Appendix C 

Data for Figure 3: How Important Students Found Various Features of the Physics 100 Open 
Textbook 


Feature 

Somewhat or 
very important 

Of average 
importance 

Somewhat or 
very unimportant 

The readings were customized to this particular 

course 

94.4% 

4.2% 

1.4% 

The Physics 100 textbook didn’t cost any money 

92.3% 

4.9% 

2.8% 

I could access the textbook anywhere with an 
internet connection 

86.0% 

11.2% 

2.8% 

The textbook could be read on my phone or other 
mobile device 

64.3% 

21.7% 

14.0% 

I could download the readings and read them 
even without an internet connection 

62.9% 

25.9% 

11.2% 

I could download the readings and print them 

49.6% 

18.9% 

31-5% 

Note: N=i43 


D Athabasca 
University 
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